A novel method to simplify decision functions of support vector machines (SVMs) is proposed in this paper. In our method, a decision function is determined first in a usual way by using all training samples. Next those support vectors which contribute less to the decision function are excluded from the training samples. Finally a new decision function is obtained by using the remaining samples. Experimental results show that the proposed method can effectively simplify decision functions of SVMs without reducing the generalization capability.

This work presents an application of the previously proposed Support Vector Machines Based Generalized Predictive Control (SVM-Based GPC) method [1] to the problem of controlling chaotic dynamics with small parameter perturbations. The Generalized Predictive Control (GPC) method, which is included in the class of Model Predictive Control, necessitates an accurate model of the plant that plays very crucial role in the control loop. On the other hand, chaotic systems exhibit very complex behavior peculiar to them and thus it is considerably difficult task to get their accurate model in the whole phase space. In this work, the Support Vector Machines (SVMs) regression algorithm is used to obtain an acceptable model of a chaotic system to be controlled. SVM-Based GPC exploits some advantages of the SVM approach and utilizes the obtained model in the GPC structure. Simulation results on several chaotic systems indicate that the SVM-Based GPC scheme provides an excellent performance with respect to local stabilization of the target (an originally unstable equilibrium point). Furthermore, it somewhat performs targeting, the task of steering the chaotic system towards the target by applying relatively small parameter perturbations. It considerably reduces the waiting time until the system, starting from random initial conditions, enters the local control region, a small neighborhood of the chosen target. Moreover, SVM-Based GPC maintains its performance in the case that the measured output is corrupted by an additive Gaussian noise.

Several research fields have to deal with very large classification problems, e.g. handwritten character recognition and speech recognition. Many works have proposed methods to address problems with large number of samples, but few works have been done concerning problems with large numbers of classes. CombNET-II was one of the first methods proposed for such a kind of task. It consists of a sequential clustering VQ based gating network (stem network) and several Multilayer Perceptron (MLP) based expert classifiers (branch networks). With the objectives of increasing the classification accuracy and providing a more flexible model, this paper proposes a new model based on the CombNET-II structure, the CombNET-III. The new model, intended for, but not limited to, problems with large number of classes, replaces the branch networks MLP with multiclass Support Vector Machines (SVM). It also introduces a new probabilistic framework that outputs posterior class probabilities, enabling the model to be applied in different scenarios (e.g. together with Hidden Markov Models). These changes permit the use of a larger number of smaller clusters, which reduce the complexity of the final classifiers. Moreover, the use of binary SVM with probabilistic outputs and a probabilistic decoding scheme permit the use of a pairwise output encoding on the branch networks, which reduces the computational complexity of the training stage. The experimental results show that the proposed model outperforms both the previous model CombNET-II and a single multiclass SVM, while presenting considerably smaller complexity than the latter. It is also confirmed that CombNET-III classification accuracy scales better with the increasing number of clusters, in comparison with CombNET-II.

In this paper, a method of detecting overlapping speech segments in meetings is proposed. It is known that the eigenvalue distribution of the spatial correlation matrix calculated from a multiple microphone input reflects information on the number and relative power of sound sources. However, in a reverberant sound field, the feature of the number of sources in the eigenvalue distribution is degraded by the room reverberation. In the Support Vector Machines approach, the eigenvalue distribution is classified into two classes (overlapping speech segments and single speech segments). In the Support Vector Regression approach, the relative power of sound sources is estimated by using the eigenvalue distribution, and overlapping speech segments are detected based on the estimated relative power. The salient feature of this approach is that the sensitivity of detecting overlapping speech segments can be controlled simply by changing the threshold value of the relative power. The proposed method was evaluated using recorded data of an actual meeting.

In this paper, Morse code is selected as a communication adaptive device for persons whose hand coordination and dexterity are impaired by such ailments as amyotrophic lateral sclerosis, multiple sclerosis, muscular dystrophy, and other severe handicaps. Morse code is composed of a series of dots, dashes, and space intervals, and each element is transmitted by sending a signal for a defined length of time. A suitable adaptive automatic recognition method is needed for persons with disabilities due to their difficulty in maintaining a stable typing rate. To overcome this problem, the proposed method combines the support vector machines method with a variable degree variable step size LMS algorithm. The method is divided into five stages: tone recognition, space recognition, training process, adaptive processing, and character recognition. Statistical analyses demonstrated that the proposed method elicited a better recognition rate in comparison to alternative methods from the literature.

A comparison of performances is made of three text-independent speaker identification methods based on dual Penalized Logistic Regression Machine (dPLRM), Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) with experiments by 10 male speakers. The methods are compared for the speech data which were collected over the period of 13 months in 6 utterance-sessions of which the earlier 3 sessions were for obtaining training data of 12 seconds' utterances. Comparisons are made with the Mel-frequency cepstrum (MFC) data versus the log-power spectrum data and also with training data in a single session versus in plural ones. It is shown that dPLRM with the log-power spectrum data is competitive with SVM and GMM methods with MFC data, when trained for the combined data collected in the earlier three sessions. dPLRM outperforms GMM method especially as the amount of training data becomes smaller. Some of these findings have been already reported in [1]-[3].

There has been significant interest in developing new forms of acoustic model, in particular models which allow additional dependencies to be represented than those contained within a standard hidden Markov model (HMM). This paper discusses one such class of models, augmented statistical models. Here, a local exponential approximation is made about some point on a base model. This allows additional dependencies within the data to be modelled than are represented in the base distribution. Augmented models based on Gaussian mixture models (GMMs) and HMMs are briefly described. These augmented models are then related to generative kernels, one approach used for allowing support vector machines (SVMs) to be applied to variable length data. The training of augmented statistical models within an SVM, generative kernel, framework is then discussed. This may be viewed as using maximum margin training to estimate statistical models. Augmented Gaussian mixture models are then evaluated using rescoring on a large vocabulary speech recognition task.

Geometrical properties of the lifting-up technique in support vector machines (SVMs) are discussed here. In many applications, an SVM finds the optimal inhomogeneous separating hyperplane in terms of margins while some of the theoretical analyses on SVMs have treated only homogeneous hyperplanes for simplicity. Although they seem equivalent due to the so-called lifting-up technique, they differ in fact and the solution of the homogeneous SVM with lifting-up strongly depends on the parameter of lifting-up. It is also shown that the solution approaches that of the inhomogeneous SVM in the asymptotic case that the parameter goes to infinity.

This paper describes a pattern classifier for detecting frontal-view faces via learning a decision boundary. The proposed classifier consists of two major parts for improving classification accuracy: the implicit modeling of both the face and the near-face classes resulting in an extended discriminative feature set, and the subsequent composite Support Vector Machines (SVMs) for speeding up the classification. For the extended discriminative feature set, Principal Component Analysis (PCA) or Independent Component Analysis (ICA) is performed for the face and near-face classes separately. The projections and distances to the two different subspaces are complementary, which significantly enhances classification accuracy of SVM. Multiple nonlinear SVMs are trained for the local facial feature spaces considering the general multi-modal characteristic of the face space. Each component SVM has a simpler boundary than that of a single SVM for the whole face space. The most appropriate component SVM is selected by a gating mechanism based on clustering. The classification by utilizing one of the multiple SVMs guarantees good generalization performance and speeds up face detection. The proposed classifier is finally implemented to work in real-time by cascading a boosting based face detector.

In this paper, we address issues in improving hands-free speech recognition performance in different car environments using multiple spatially distributed microphones. In the previous work, we proposed the multiple linear regression of the log spectra (MRLS) for estimating the log spectra of speech at a close-talking microphone. In this paper, the concept is extended to nonlinear regressions. Regressions in the cepstrum domain are also investigated. An effective algorithm is developed to adapt the regression weights automatically to different noise environments. Compared to the nearest distant microphone and adaptive beamformer (Generalized Sidelobe Canceller), the proposed adaptive nonlinear regression approach shows an advantage in the average relative word error rate (WER) reductions of 58.5% and 10.3%, respectively, for isolated word recognition under 15 real car environments.

An efficient, scalable speech recognition architecture combining topic detection and topic-dependent language modeling is proposed for multi-domain spoken language systems. In the proposed approach, the inferred topic is automatically detected from the user's utterance, and speech recognition is then performed by applying an appropriate topic-dependent language model. This approach enables users to freely switch between domains while maintaining high recognition accuracy. As topic detection is performed on a single utterance, detection errors may occur and propagate through the system. To improve robustness, a hierarchical back-off mechanism is introduced where detailed topic models are applied when topic detection is confident and wider models that cover multiple topics are applied in cases of uncertainty. The performance of the proposed architecture is evaluated when combined with two topic detection methods: unigram likelihood and SVMs (Support Vector Machines). On the ATR Basic Travel Expression Corpus, both methods provide a significant reduction in WER (9.7% and 10.3%, respectively) compared to a single language model system. Furthermore, recognition accuracy is comparable to performing decoding with all topic-dependent models in parallel, while the required computational cost is much reduced.

The choice of kernel is an important issue in the support vector machine algorithm, and the performance of it largely depends on the kernel. Up to now, no general rule is available as to which kernel should be used. In this paper we investigate two kernels: Gaussian RBF kernel and polynomial kernel. So far Gaussian RBF kernel is the best choice for practical applications. This paper shows that the polynomial kernel in the normalized feature space behaves better or as good as Gaussian RBF kernel. The polynomial kernel in the normalized feature space is the best alternative to Gaussian RBF kernel.

This study aims to identify grammatical relations (GRs) in Korean sentences. The key task is to find the GRs in sentences in terms of such GR categories as subject, object, and adverbial. To overcome this problem, we are faced with the structural ambiguity and the grammatical relational ambiguity. We propose a statistical model, which resolves the grammatical relational ambiguity first, and then resolves structural ambiguity by using the probabilities of the GRs given noun phrases and verb phrases in sentences. The proposed model uses the characteristics of the Korean language such as distance, no-crossing and case property. We showed that consideration of such characteristics produces better results than without consideration by experiments. We attempt to enhance our system by estimating the probabilities of the proposed model with the Maximum Entropy (ME) model, and with Support Vector Machines (SVM) classifiers and we confirm that SVM classifiers improved the performance of our proposed model through experiments. Through an experiment with a tree and GR tagged corpus for training the model, we achieved an overall accuracy of 84.8%, 94.1%, and 84.8% in identifying subject, object, and adverbial relations in sentences, respectively.

Masqueraders who impersonate other users pose serious threat to computer security. Unfortunately, firewalls or misuse-based intrusion detection systems are generally ineffective in detecting masqueraders. Anomaly detection techniques have been proposed as a complementary approach to overcome such limitations. However, they are not accurate enough in detection, and the rate of false alarm is too high for the technique to be applied in practice. For example, recent empirical studies on masquerade detection using UNIX commands found the accuracy to be below 70%. In this research, we performed a comparative study to investigate the effectiveness of SVM (Support Vector Machine) technique using the same data set and configuration reported in the previous experiments. In order to improve accuracy of masquerade detection, we used command frequencies in sliding windows as feature sets. In addition, we chose to ignore commands commonly used by all the users and introduce the concept of voting engine. Though still imperfect, we were able to improve the accuracy of masquerade detection to 80.1% and 94.8%, whereas previous studies reported accuracy of 69.3% and 62.8% in the same configurations. This study convincingly demonstrates that SVM is useful as an anomaly detection technique and that there are several advantages SVM offers as a tool to detect masqueraders.

This paper proposes a learning classifier based on Support Vector Domain Description (SVDD) for two-class problem. First, by the description of the training samples from one class, a sphere boundary containing these samples is obtained; then, this boundary is used to classify the test samples. In addition, instead of the traditional quadratic programming, multiplicative updates is used to solve the Lagrange multiplier in optimizing the solution of the sphere boundary. The experiment on CBCL face database illustrates the effectiveness of this learning algorithm in comparison with Support Vector Machine (SVM) and Sequential Minimal Optimization (SMO).

This paper presents a local learning framework in which the local classifiers can be pre-learned and the support size of each classifier can be selected to minimize the error bound. The proposed algorithm is compared with the conventional support vector machine (SVM). Experimental results show that our scheme using the user-defined parameters C and σ is more accurate and less sensitive than the conventional SVM.

Support vector algorithms try to maximize the shortest distance between sample points and discrimination hyperplane. This paper suggests the total margin algorithms which consider the distance between all data points and the separating hyperplane. The method extends and modifies the existing algorithms. Experimental studies show that the total margin algorithms provide good performance comparing with the existing support vector algorithms.

Although many researchers have verified the superiority of Support Vector Machine (SVM) on text categorization tasks, some recent papers have reported much lower performance of SVM based text categorization methods when focusing on all types of parts of speech (POS) as input words and treating large numbers of training documents. This was caused by the overfitting problem that SVM sometimes selected unsuitable support vectors for each category in the training set. To avoid the overfitting problem, we propose a two step text categorization method with a variable cascaded feature selection (VCFS) using SVM. VCFS method selects a pair of the best number of words and the best POS combination for each category at each step of the cascade. We made use of the difference of words with the highest mutual information for each category on each POS combination. Through the experiments, we confirmed the validation of VCFS method compared with other SVM based text categorization methods, since our results showed that the macro-averaged F1 measure (64.8%) of VCFS method was significantly better than any reported F1 measures, though the micro-averaged F1 measure (85.4%) of VCFS method was similar to them.

Kernel methods such as the support vector machines map input vectors into a high-dimensional feature space and linearly separate them there. The dimensionality of the feature space depends on a kernel function and is sometimes of an infinite dimension. The Gauss kernel is such an example. We discuss the effective dimension of the feature space with the Gauss kernel and show that it can be approximated to a sum of polynomial kernels and that its dimensionality is determined by the boundedness of the input space by considering the Taylor expansion of the kernel Gram matrix.

In face recognition, simple classifiers are frequently used. For a robust system, it is common to construct a multi-class classifier by combining the outputs of several binary classifiers; this is called output coding method. The two basic output coding methods for this purpose are known as OnePerClass (OPC) and PairWise Coupling (PWC). The performance of output coding methods depends on accuracy of base dichotomizers. Support Vector Machine (SVM) is suitable for this purpose. In this paper, we review output coding methods and introduce a new sequential fusion method using SVM as a base classifier based on OPC and PWC according to their properties. In the experiments, we compare our proposed method with others. The experimental results show that our proposed method can improve the performance significantly on the real dataset.